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ABSTRACT 



The basic questions which we tried to answer in this paper, concern the 
generalizibility of effective schools indicators across contexts. We attempt to 
establish which characteristics of school effectiveness are context-bound and 
which seem to be more generally valid. A further explicit aim was to relate 
school-effectiveness research more closely to effectiveness perspectives 
developed in the field of organizational theory. 
The effective schools movement claims that a limited set of school 
characteristics consistently co-variates with the educational attainments of the 
students. The empirical evidence for these claims is. however, much debated. 
First of all there is evidence that schools that are effective one year are not 
effective anymore the next year. Secondly, some studies show that effective 
schools characteristics variate across contexts, e.g. primary vs. secondary 
schools, private ys. public schools and schools with high- vs. low ability 
students. In this study an inventarisation has been made of reliable effective 
schools characteristics within as well as across contexts, including a 
comparison of results of effective schools research from different nations. 
Schooleffectiveness is seen in terms of multi-level causal network of context-, 
school organisation- and individuaTstrategies. Furthermore this "contextual 
analysis" of schooleffectiveness is compared with perspectives from 
organization theory, most notably the contingency-theory. 
The results of this study show cleariy that it is not appropriate to speak of 'the' 
characteristics of effective schools. These characteristics vary considerably 
across contexts and nations. On the other hand some evidence remains that 
indicates that some characteristics are valid across contexts. The evaluative 
potential of a school is one of them.The context-boundneness of 
characteristics of effective schools is explained by cultural differences between 
contexts and by the organizational differences between schools in different 
contexts. 

It is most likely that what 'works' in one context does not always work in 
another. There needs to be a 'fit' between context, organisation and in- school 
measures to obtain higher effectivity (and efficiency). 



These conclusions show that the general idea of contingency-theory also 
applies to effective schools-research. Moreover a variety of other organization- 
theoretic concepts should be employed more fully in future school 
effectiveness research. 



1. INTRODUCTION 



At present we discern two new trends in the school-effectiveness literature. 
Firstly, in research syntheses of educational productivity research we see that 
both indicators at the school (or meso-) level and indicators at the classroom 
(or micro-) level are presented together. The idea of combining school- and 
classroom-indicators in effectiveness studies corresponds with Purkey and 
Smith's idea of conceptualizing school effectiveness conditions as "nested 
layers" (Purkey and Smith, 1983). The developments in multi-level statistical 
analysis (e.g. Raudenbush and Bryk, 1985; Aitkin & Longford, 1986) are 
particularly relevant to school effectiveness studies in which one wishes to 
estimate simultaneous'y school and classroom effects, although there is still a 
gap between the complexity of the conceptual schemes and what the 
techniques can handle (Scheerens & Steel, 1987). 
The second more recent development in school effectiveness research 
elaborates the conceptual scheme of school effectiveness further by adding 
contextual conditions of schools (Wimpelberg, Teddii© & Springfield, 1987; 
Teddlie, Springfield & Wimpelberg, 1987). Examples of school-context 
variables are: private vs. public schools, SES-compositlon and primary vs. 
secondary schools. Although these factors are somcatlmes referred to as 
organizational variables, we feel that contextual variables Is the better term 
since they lie only partly within the managerial sphere of influence of the 
individual school, a phenomenon which is even more pronoui when we 
include characteristics of educational policy^at a higher level w . tne school 
(district, state, nation). 

As Wimpelberg et al. (1987) point out. the "5-factor modeP of school 
effectiveness has been empirically verified almost exclusively in primary 
schools in urban contexts with a lower SES-student composition. Teddlie et al. 
(1987) provide research results from which they induce different effectiveness 
conditions for Middle-SES-schools as composed to lower SES-schools. 
Firestone & Herriott (1982) show that effectiveness conditions found in primary 
schools cannot simply be generalized to secondary schools. 
In this paper we shall present some material that bears upon the contextual 
dimension of school effectiveness. First, we discuss the results of a 
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comparison of effectiveness indicators across two types of secondary 
education in the Netherlands, the LBO- (lower vocational education) and the 
MAVO- (lower general secondary education) schools. In a second section we 
draw a general comparison between the main findings of school effectiveness 
research in the Netherlands and the major ANGLO-Saxon effectiveness 
studies, and, in addition, look more specifically at two Dutch replications of 
American studies. 

In the final section we summarize tentative explanations for the differences 
between Dutch and Anglo-Saxon school effectiveness research-findings. 

2. THE VALIDITY OF THE 5-FACTORMODEL 

Since there exist already an abundance of summary-articles on school- 
effectiveness and educational productivity research (e.g. Purkey and Smith, 
1983; Ralph & Fenessey, 1983; Putter, 1983; Walberg, 1984), we shall be brief 
in sketching the developments in this field of inquiry. The shift in emphasis in 
effectiveness research from input-output, to process-output studies, also 
referred to as the first and second phase of effectiveness research, is well 
documented in the literature, where Coleman et al. (1 966) and Jencks et al. 
(1972) are representative for phase one, and Edmonds, 1979; Brookover, 1 979 
and Rutter, 1 979, are important studies in the phase two tradition. This latter 
research tradition has yielded, what is sometimes referred to as a "5-factor 
model of school effectiveness": academic goal consensus, safe and orderiy 
climate, strong instructional '?adership, high expectations for student 
achievement and frequent evaluation of pupil progress. 

There seems to exist considerable consensus about the validity of this set of 
school characteristics. Almost every review of research on effective schools 
claims the factors mentioned above to be effective. Apparently one assumes 
that this set of variables 'explains' most of the variance between student 
outcomes which are not accounted for by individual differences between 
students. However there should be some doubt about this proposition. 
Firstly most research on effective schools has been carried out in the United 
States. One should wonder whether results of this research can be transferred 
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to Other counti .3s. Different countries can have different schoolsystems and 
different schoolcultures. Therefore it is likc.'y that the characteristics of effective 
schools are different from one country to another. In (a next) paragraph we will 
present some empirical evidence for this proposition. 
Secondly another assumption that is frequently implicitly made is the 
assumption that these so-called effective school characteristics have a positive 
influence on all students, regardless of their socio-economic background, 
abilitylevel or there ethnic background. It is debatable whether this is indeed 
the case. 

Thirdly, one should take into account that most of the research done on 
effective schools has taken place in primary schools. Nevertheless in research- 
papers one seldom refers to the limits in generalizibility of results to, for 
instance, secondary schools. 

A fourth reason to question the generalizibility of the so-called 5-factor model is 
the difference between rural and urban area's. Youth culture is different for 
urban and rural area's, so one could assume that characteristics of effective 
schools are also different for different area's. 

In the next paragraphes we trie to present some empirical evidence for the 
propositions mentioned above. 

3. OVERVIEW OF DUTCH EFFECTIVENESS STUDIES 
3.1. The dutch educational system 

As we mentioned before, differences between schoolsystems in different 
countries could result in different sets of effective schoolcharacteristics. 
Therefore we will present in short the main characteristics of the Dutch 
educational system. 

There are three characteristics of the Dutch school system that should be 
mentioned if the research is to be understood. 

One is that secondary education is split up into at least four different streams, 
each almost isolated from the other in a different school type. There has been 
much political debate about changing the system and "going more 
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comprehensive", but the different schooltype system is likely to exist for at 
least another decade. 



insert figure 1 here 



At the grade levels at which the research that is to be presented in paragraph 
3.2. took place, an important difference exists between general education 
(mavo and havo/vwo), taking about 60% of the age group and junior vocational 
education (Ihno and Ibo) taking about 30% of the age group. The higher type of 
general education, havo/vwo, gives access to the university and a number of 
professional schools. Havo/vwo differs from the other school types in terms of 
the number of years of study and in terms of the curriculum. Teachers in these 
schools are also better qualified and better paid. Many of them have Master's 
degrees in their subject. Lhno is mainly for girls and Ibo mainly for boys, at 
least in our studies. The above school types exist work as separate institutions 
from the 7th grade onwards. 

The second characteristic to be mentioned is that the Dutch educational 
system is centralized for funding and final examinations. In some ways, this 
national centralization prohibits any strong influence by local authorities. Of 
course, this centralization could have an impact on the differences that exist 
between schools of the same type. One could assume that differences within 
each type of school are smaller in the Netherlands than for instance in the 
United States, where, there is less centralization. 

3.2. Overview of Dutch effectiveness studies 

In the Netherlands, relatively few school effectiveness-studies have been 
conducted, as yet. Moreover, these studies were carrad out during the last few 
years, so there is every reason to say that we are dealing with a research field 
in its infancy. 

An interesting development, however, is the fact that the more mature field of 
school career research in the Netherlands is gradually blending with school 
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were supposed to cover five characteristics of effective schools: instructional 
leadership, emphasis on achievement, a safe and orderly climate, high 
expectations and use of assessment instruments. The correlations that were 
reported by Schweitzer were relatively and therefore ve 7 promising for schools 
who want to improve their output. 



Insert table 2 here 



In table 2 and 3 the results of the Schweitzer-study are summarized. 
Vermeulen replicated the Schweitzer-study in the Netherlands in 1986. 
Vermeulen carefully translated the questionnaire used by Schweitzer. 

Insert table 3 here 



Table 2 and 3 also show the results of this study by Vermeulen. As you can 
see the correlations that were reported by Vern 9ulen are much smaller than 
those reported by Schweitzer. Table 3 even shovs a reverse of the Schweitzer 
results. Whereas Schweitzer reported positive correlations, Vermeulen found 
negative correlations. Theoretically there are many explanations for these 
differences. Cultural differences between two countries coula be one of them: 
what apparently works in the USA should not always work in other countries 
toe. On the other hand it is very plausible that rhe same situation could occur 
inside the United States: if a study is carried 0 't in one place in a specific 
context, it is not very likely that the same results are to be found in other 
districts, schools etc. The conclusion should be that it is necessary to conduct 
more replicationstudies, in order to establish whether there are contextual 
influences in effective schools characteristics. 
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3.3.2. Replication of the findings by Sweeney (USA) by Van der Grift (NL) 

In 1982 Sweeney made a synthesis on research on effective schoolleadership. 
He compared 8 studies in this field. As a result of this synthesis he found six 
categories of leadership behavior to be positively associated with school 
outcome, in table 4 the results reported in his review are summarized. 

insert table 4 here 



Table 4 shows clearly that Wiiere Sweeney concludes that these six categories 
of schoolleadership behavior are strongly related to school outcome, Van der 
Grift cannot reproduce these relations in a Dutch educational setting. The only 
significant correlation he found is even negatively correlated to school 
outcome. Again there are many possible reasons why this is the case. 
Nevertheless, one should take into consideration the possibility that contextual 
effects are part of an explanation of these somewhat confusing results. 

3.4. Comparison of two different types of secondary education 

In 1982 Stoel (1986) did some research of schooleffects in secondary 
education in the Netherlandfj. In this research two types of secondary 
education (vgl. par. 3.1 .) were involved: schools for lower general education 
MAVO and schools for [jwer vocational education (LBO). 
Information about schoolcharacteristics were gathered by means of a 
questionaire, that was answered by the schoolleader. Schoollevel-information 
was obtained from context-, input-, process- and output-variables. The 
selection of variables had taken place on the basis of research-literature and 
interviews with schoolleaders, policymakers and teachers. Variables which 
were supposed to explain most variance in students-output at the schoollevel 
were selected. 

The two samples were randomly selected from each sub-population (schools 
for lower general education (MAVO) resp. schools for lower vocational 
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education (LBO)). Eventually 1 32 MAVO-and 119 LBO-schools participated in 
the survey. 

With respect to several output-measures data were gathered: percentage of 
students that pass exams, percentage of students that repeated classes in 
each grade, percentage of students that dropped out, percentage of students 
that were truant and some more. Simple PM-correlatlons and regression- 
analysis were used to establish the relationship between input, context and 
processvariables on one hand and the output-variables on the other hand. 
Only the correlations with dropout and truancy-rates are presented here. There 
are two reasons for this selection. The percentago of students that passes 
exams could not be predicted by what schoolcharacteristic whatsoever. 
Apparently there was no consistant co-variation between exim-results and 
schoolcharacteristics. Secondly, the remaining output-measures could be 
predicted by schoolcharacteristics in some way, but relationships were very 
small. Only dropout and truancy-rates could be predicted by 
schoolcharacteristics to a more substantial degree. In table 5 only the 
significant correlations are presented for both LBO- and MAVO-schools 
between school dropout rates and some schoolcharacteristics. 

insert tables 5 and 6 here 



As you can see the correlations are not only small, they are also very 
inconsistent for both type of school^ When an indicator/criterion relationship 
was established for one school-type, it was not reproduced for the other; in 
some schools even negative correlations were found. One could argue that the 
pattern of correlations shown above is merely the result of unreliable 
Instruments or chance capitalisation. Because the reliability of 
schoollevelm asures is mostly unknown and chance capitalisation can never 
be exclud'-d loially. It is difficult to fight these arguments. If one assumes that 
the correlations presented above are real, one can easily see what 'works' for 
one type school does not necessarily works for another. If one assumes that 
these correlations are not real but due to unreliable measures and chance 



capitalisation, then one should conclude thrt as far as the Netherlands are 
concerned, there is no relation whatsoever between schoolcharacteristics, that 
are said to le effective according to anglo-american studies, and student 
dropout-rates. Both conclusions lead to a third one. Apparently 
schoo contextual characteristics, e.g. type of school and nation, affect what 
characteristics of schools are related to student-output. 

3.5.Dutch replications of foreign effectiveness studies 

One possible explanation for the lack of corraboration of the "five-factor model" 
in Dutch school effectiveness research, could be the fact that, the variance 
between schools of a particular schooltype, is lower than in the Anglo-Saxon 
countries. Roeleveld (1987) found that 7-13% of the variance in educational 
attainment of pupils, lay between schools. Stoel, 1986 found that this 
percentage was 6 to 1 7% within different types of secondary education. 
Whether this should be considered high or low can only be induced from 
careful international comparison of achievement data. (For this reason we are 
presently involved in a secondary analysis of lEA-data). 
Although schools in the Netheriands have a lot of autonomy, as far as the 
choice of curricula and teaching methods are concerned, there is relative 
uniformity with respect to teacher salaries, job differentiation that is allowed, 
and resources. 

Two other characteristics could explain the inconsistency in findin^js across 
studies in measures like "frequent evaluation' and 'achievement cientation'. 
Although we lack empirical data on this subject, our personal impression is that 
achievement orientation does not generally rank high in the teaching 
philosophy of most Dutch schools. 

The humanistic and reform pedagogic ideals of the seventies, which are at 
odds with high achievement orientation with respect to basic skills still have a 
lot of influence. Nor is there a testing and evali;ati'^n tradition in Dutch schools. 
Only quite recently preparations are being made for a national assessment 
program. One of its envisaged spin-offs, namely instruments for self-evaluation 
of schools is still further away. 



So, a certain uniformity in the Dutch school system could explain relatively low 
overall school effects and lack of achievement orientation and testing and 
evaluation triaition could explain relatively little variance in some of the 
independent variables of school effectiveness research. Together these 
phenomena could explain the very weak corroboration of the "five-factor 
model" in Dutch school effectiveness research. 



4. CONCLUSION 



The results of this study show clearly that it Is not appropriate to speak of 'the' 
characteristics of effective schools. These characteristics vary considerably 
across contexts and nations. On the other hand some evidence remains that 
indicates that some characteristics are valid across contexts. The evaluative 
potential of a school is one of them.The context-boundneness of 
characteristics of effective schools is explained by cultural differences between 
contexts and by the organizational differences between schools in different 
contexts. 

!n a recent review of Fuller (1987) he compared results of research on effective 
schools that has been carried out in the so-called third world countries. One of 
his conclusions, amongst others which are also very noteworthy, is that school 
effects in the Third World seem to be stronger in rural aeiaa's and among 
lower income pupils. 

Again there is a clear evidence that contextual effects should be taken into 
account. Not only when one reports on research on effective schools but als 
when one tries to improve schools on the basis of this kind of research. In our 
opinion this counts not only for contextual diferences between countries but 
also for contextual differences between schools inside countries. 
It is most likely that what 'works' in one context does not always work in 
another. There needs to be a 'fit' between context, organisation and in- school 
measures to obtain higher effectivity (and efficiency). 
These conclusions show that the general idea of contingency-theory also 
applies to effective schools-research. Moreover a variety of other organization- 
theoretic concepts should be employed more fully in future school 
effectiveness research. 
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One final conclusion we would like to draw from the results that have been 
presented is the apparent necessity to take contextual conditions into account 
when conducting and analyzing school effectiveness research. Perhaps this is 
not a pleasant consideration when our aim is a parsiminous school 
effectiveness model. But only by testing hypothetical sets of school 
effectiveness indicators across contexts, such as school-types, districts and 
even nations, can we find out which are generalizible and which are context- 
bound. 

We feel that a multiple leve! model of school effectiveness should be born in 
mind as a conceptu al background of individual studies - not that each study 
should try and cover the full complexity of the model. The theoretical 
implications of this model are discussed by Scheerens & Stoel, 1988. 
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Figure 1 : The educational system for secondary education in the Netherlands 
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Table 1 : Overview of Dutch effectiveness studies 
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Table 2: Relationship between characteristics of schools and educational 
attainment (reported by teachers). 

Schweitzer (USA) Vermeulen (NL) 



Strong instruction leadership .58 .05 

Academic goal consensus .79 .12 

Safe and orderly climate .59 .34 

High expectations for student .79 .21 
achievement 

Frequent evaluations of pupil .68 .23 
progress 



Table 3; Relationship between characteristics of schools and educational 
attainment (reported by schoolleader). 



Schweitzer (USA) Vermeulen (NL) 



Strong instruction leadership .09 -.56 

Academic goal consensus .44 -.19 

Safe and orderly climate .28 .21 

High expectations for student .71 -.45 
achiever i 

Frequent evaluations of pupil .21 .13 
progress 



Table 4: Comparison of the outcomes of the review of studies reported by 
Sweeney with a research study carried out by Van der Grift 



Sweeney Van der Grift 

(N=139) 

category of leadership 
behavior 



1 . coordinates instructional 4* 

2. emphasizes achievement 8 

3. frequent evaluation 5 
of pupil progress 

4. provides orderly 7 
atmosphere 

5. sets instructional 8 
strategies 

6. supports teachers 3 



* Number of studies (out of 8) that reported positive associations with school 
outcome 

" significant P.M.-correlation at the .05 level 



-.14" 

n.s. 

n.s. 

n.s. 

n.s. 

n.s. 



Table 5: P.M.-correlations between several school (process) variables and the 
dropout-rate of schools 



lower vocational lower general 
education (N=119) education (N=132) 



school (process) variable 



pressure on homework -.45 n.s. 

frequency teachers .23 -.24 
meeting in sections 

individualized instruction .21 n.s. 

time spent on meetings .31 n.s. 
by school management 

pupil/teacher ratio .30 .29 

average class-size -.39 -.26 

percentage of students .20 .53 
from ethnic minorities 



Table 6: P.M.-correlations between several school (process) variables and the 
truancy-rate of schools 



school (process) variable 



lower vocational lower general 
education (N=1 1 9) education (N=1 32) 



frequency of extra- n.s. .20 

curricular activities 

intensity of use of -.23 n.s. 

diagnostic tests 



intensity of .23 
counseling activities 



n.s. 



. age of schoolbuilding -.21 n.s. 

. homework pressure n.s. -.26 

. intensity of use of n.s. .34 
evaluation practices 
by teachers 

intensity information n.s. -.29 
towards parents 



